CytoAtlas

The key insight: CytoAtlas is not trying to replace DL-based tools. It provides an orthogonal, complementary signal that a human scientist can directly inspect. When CytoAtlas says "IFNG activity is elevated in CD8+ T cells from RA patients," you can verify this by checking the IFNG signature genes in those cells.

How “N Targets” is determined: A target is included in the validation for a given atlas only if (1) the target’s signature genes overlap sufficiently with the atlas gene expression matrix, and (2) the target gene itself is expressed in enough samples to compute a meaningful Spearman correlation. Targets whose gene is absent or not detected in a dataset are excluded.

Donor-only atlases (CIMA, Inflammation, GTEx, TCGA): N = number of unique targets with valid correlations. CytoSig defines 43 cytokines and SecAct defines 1,170 secreted proteins. The Inflammation Atlas (main/validation cohorts) retains only 33 of 43 CytoSig targets and 805 of 1,170 SecAct targets because 10 cytokine genes (BDNF, BMP4, CXCL12, GCSF, IFN1, IL13, IL17A, IL36, IL4, WNT3A) are not sufficiently expressed in these blood/PBMC samples. CIMA, GTEx, and similar multi-organ datasets retain nearly all targets (≥97%).

Donor-organ atlases (scAtlas Normal, scAtlas Cancer): N = target × organ pairs, because validation is stratified by organ/tissue context. For scAtlas Normal, each target is validated independently across 25 organs (Bladder, Blood, Breast, Colon, Heart, Kidney, Liver, Lung, etc.), yielding up to 43 × 25 = 1,075 CytoSig entries (actual: 1,013 after filtering) and 1,140 × 25 = 28,500 SecAct entries (actual: 27,154). For scAtlas Cancer, validation spans 7 tissue contexts (Tumor, Adjacent, Blood, Metastasis, Pleural Fluids, Pre-Lesion, All), yielding 43 × 7 = 301 CytoSig entries (actual: 295) and 1,140 × 7 = 7,980 SecAct entries (actual: 7,809). Some target-organ pairs are excluded when the target gene lacks sufficient expression in that organ.

Note on scAtlas duplicate entries: At finer aggregation levels (e.g., donor_organ_celltype1 vs donor_organ_celltype2), the same target can appear multiple times with different correlation values. This is expected — finer cell-type annotation changes the composition of each pseudobulk sample, yielding different expression-activity relationships. The summary table above uses the donor_organ level for scAtlas.

Why does SecAct appear to underperform CytoSig in the Inflammation Atlas?

This is a composition effect, not a genuine performance gap. CytoSig tests only 43 curated, high-signal cytokines, while SecAct tests 1,170 secreted proteins — including many tissue-expressed targets (collagens, metalloproteinases, apolipoproteins, complement factors) with minimal expression variation in blood/PBMC samples. On the 22 matched targets shared between both methods, SecAct consistently outperforms CytoSig across all atlases (e.g., median ρ = 0.51 vs 0.32 in Inflammation Main).

The Inflammation Atlas is largely blood-derived, so many SecAct targets that perform well in multi-organ contexts (scAtlas, GTEx, TCGA) contribute near-zero or negative correlations here. In fact, 99 SecAct targets are negative only in inflammation but positive in all other atlases, reflecting tissue-specific expression limitations rather than inference failure. The “SecAct (CytoSig-matched)” boxplot above demonstrates the fair comparison on equal footing.

Key insight: None of these targets have isoforms or subunits mapping to different gene IDs that would cause gene ID confusion. The poor correlations are driven by post-translational regulation (membrane shedding, proteolytic activation, decoy receptor sequestration), paracrine signaling topology (producer and responder cells are different cell types), and heteromeric complex dependence (LTA requires LTB). These represent fundamental limitations of using ligand mRNA abundance to predict downstream signaling activity — the CytoSig activity scores themselves remain valid readouts of pathway activation in the measured cells.

Aggregation levels explained: Pseudobulk profiles are aggregated at increasingly fine cell-type resolution. At coarser levels, each pseudobulk profile averages more cells, yielding smoother expression estimates but masking cell-type-specific signals. At finer levels, each profile is more cell-type-specific but based on fewer cells.

How each correlation value is computed: For each (target, atlas) cell, we compute Spearman rank correlation between predicted cytokine activity (ridge regression z-score) and target gene expression across all donor-level pseudobulk samples. Specifically:

1. Pseudobulk aggregation: For each atlas, gene expression is aggregated to the donor level (one profile per donor or donor × cell type).
2. Activity inference: Ridge regression (secactpy.ridge, λ=5×10⁵) is applied using the signature matrix (CytoSig: 4,881 genes × 43 cytokines; SecAct: 7,919 genes × 1,170 targets) to predict activity z-scores for each pseudobulk sample.
3. Correlation: Spearman ρ is computed between the predicted activity z-score and the original expression of the target gene across all donor-level samples within that atlas. A positive ρ means higher predicted activity tracks with higher target gene expression.

GTEx/TCGA use donor-only pseudobulk; CIMA uses donor-only; Inflammation uses donor-only; scAtlas uses donor × organ.

Ten methods compared on identical matched pairs across 4 combined atlases:

1. CytoSig — 43 cytokines, 4,881 curated genes, global (all cell types)
2. LinCytoSig (orig) — cell-type-matched signatures, all ~20K genes
3. LinCytoSig (gene-filtered) — cell-type-matched signatures, restricted to CytoSig’s 4,881 genes
4. LinCytoSig Best (combined) — best cell-type signature per cytokine (selected by combined GTEx+TCGA bulk ρ), all ~20K genes
5. LinCytoSig Best (comb+filt) — best combined bulk signature, restricted to 4,881 genes
6. LinCytoSig Best (GTEx) — best per cytokine selected by GTEx-only bulk ρ, all ~20K genes
7. LinCytoSig Best (TCGA) — best per cytokine selected by TCGA-only bulk ρ, all ~20K genes
8. LinCytoSig Best (GTEx+filt) — GTEx-selected best, restricted to 4,881 genes
9. LinCytoSig Best (TCGA+filt) — TCGA-selected best, restricted to 4,881 genes
10. SecAct — 1,170 secreted proteins (Moran’s I), subset matching CytoSig targets

Key findings:

• SecAct achieves the highest median ρ across all 4 combined atlases, benefiting from spatial-transcriptomics-derived signatures.
• CytoSig outperforms most LinCytoSig variants at donor level, with one notable exception: scAtlas Normal Best-orig (0.298) exceeds CytoSig (0.216).
• Gene filtering improves LinCytoSig in most atlases (CIMA +102%, Inflammation Atlas), confirming noise reduction from restricting the gene space.
• GTEx-selected best performs comparably to combined-selected in most atlases but slightly better in scAtlas Cancer (0.300 vs 0.275). TCGA-selected best generally underperforms other selection strategies, suggesting GTEx’s broader tissue coverage provides more generalizable selections.
• Gene filtering of GTEx/TCGA-selected: GTEx+filt and TCGA+filt show mixed results — filtering sometimes improves (e.g., TCGA+filt in Inflammation Atlas: 0.260 vs TCGA-orig 0.168) but can also reduce performance, indicating the optimal gene space depends on both the selection dataset and atlas context.
• General ranking: SecAct > CytoSig > LinCytoSig Best variants > LinCytoSig (filt) > LinCytoSig (orig), though atlas-specific exceptions exist.

Key finding: Lineage-specific aggregation provides no systematic advantage at any level.

• At every level, LinCytoSig underperforms CytoSig (mean Δρ ranges from −0.08 at coarse L1 to −0.02 at fine L4 in CIMA). Finer cell types reduce the gap slightly but never close it.
• SecAct wins at every level in CIMA and scAtlas. In Inflammation Atlas L2, LinCytoSig is nearly tied with SecAct (Δρ = +0.01) but still loses to CytoSig.
• Per cell type: Only 5 of 43 cell types show consistent LinCytoSig advantage vs CytoSig (NK Cell, Basophil, DC, Trophoblast, Arterial Endothelial). No cell type beats SecAct.
• Interpretation: CytoSig’s global signature, derived from median log2FC across all cell types, already captures the dominant transcriptional response. Restricting to a single cell type’s response introduces noise from small sample sizes without gaining meaningful lineage specificity. The hypothesis that finer resolution should favor LinCytoSig is not supported by the data.